Crowd-sourced operational metric analysis of virtual appliances

ABSTRACT

A system and method for performing an operational metric analysis for a virtual appliance uses application operational data from multiple instances of the virtual appliance. The application operational data is then used to generate an operational metric prediction for the virtual appliance.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims the benefit of U.S.patent application Ser. No. 14/316,695, entitled “CROWD-SOURCEDOPERATIONAL METRIC ANALYSIS OF VIRTUAL APPLIANCES,” and filed Jun. 26,2014, which is hereby incorporated by reference in its entirety.

BACKGROUND

Operational metric analysis techniques for computer systems withresource-consuming clients, such as virtual machines (VMs), areimportant to ensure that the clients are operating at desired or targetlevels. Virtual appliances or virtual applications (VAs), which arepre-packaged virtual machine images, can be run on variousvirtualization platforms and used for public, private and hybrid cloudenvironments. For example, virtual appliances include softwarecomponents/stacks along with metadata about their anticipated aggregateresource requirements, e.g., amount of memory and/or number of processorfrequency desired for the virtual appliances. Accurate estimates ofresource requirements of virtual appliances can both influence resourcesettings, such as number of processors and amount of memory, of virtualappliances. Allocating insufficient resources to a virtual appliance canpotentially impact the performance, reliability and stability of thevirtual appliance, while allocating excessive resources to a virtualappliance is wasteful. In addition, accurate estimates of performancecharacteristics (e.g., latency and throughout) of virtual appliances caninfluence the deployment of virtual appliances.

Predicting or estimating resource usage and/or performancecharacteristics of a virtual appliance is a challenging task. Componentinteractions and application complexity can result in complex,non-linear relationships between virtual appliance performance/behaviorand resource usage. In addition, the amount of data related to resourceusage and/or performance characteristics of a virtual appliance can beenormous. Therefore, there is a need for an operational metric analysisof virtual appliances that can efficiently provide effective operationalmetric predictions for virtual appliances.

SUMMARY

A system and method for performing an operational metric analysis for avirtual appliance uses application operational data from multipleinstances of the virtual appliance. The application operational data isthen used to generate an operational metric prediction for the virtualappliance.

A method for performing an operational metric analysis for a virtualappliance in accordance with an embodiment of the invention includesobtaining application operational data from multiple instances of thevirtual appliance and generating an operational metric prediction forthe virtual appliance based on the application operational data. In someembodiments, the steps of this method are performed when programinstructions contained in a computer-readable storage medium is executedby one or more processors.

A system for performing an operational metric analysis for a virtualappliance includes a processor and an operational metric analysis systemoperably connected to the processor. The operational metric analysissystem is configured to obtain application operational data frommultiple instances of the virtual appliance and generate an operationalmetric prediction for the virtual appliance based on the applicationoperational data.

Other aspects and advantages of embodiments of the present inventionwill become apparent from the following detailed description, taken inconjunction with the accompanying drawings, illustrated by way ofexample of the principles of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts an operational metric analysis system in accordance withan embodiment of the invention.

FIG. 2 is a block diagram of a distributed computer system in accordancewith an embodiment of the invention.

FIG. 3 is a block diagram of a host computer in accordance with anembodiment of the invention.

FIG. 4 is a block diagram of components of the operational metricanalysis system in accordance with an embodiment of the invention.

FIG. 5 illustrates a list of categories of operational features ofmultiple instances of a virtual appliance.

FIG. 6 is a flow chart that illustrates an exemplary operation of theoperational metric analysis system depicted in FIG. 4.

FIG. 7 is a flow diagram of a method for performing an operationalmetric analysis for a virtual appliance in accordance with an embodimentof the invention.

Throughout the description, similar reference numbers may be used toidentify similar elements.

DETAILED DESCRIPTION

It will be readily understood that the components of the embodiments asgenerally described herein and illustrated in the appended figures couldbe arranged and designed in a wide variety of different configurations.Thus, the following more detailed description of various embodiments, asrepresented in the figures, is not intended to limit the scope of thepresent disclosure, but is merely representative of various embodiments.While the various aspects of the embodiments are presented in drawings,the drawings are not necessarily drawn to scale unless specificallyindicated.

The present invention may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. The scope of the invention is, therefore, indicatedby the appended claims rather than by this detailed description. Allchanges which come within the meaning and range of equivalency of theclaims are to be embraced within their scope.

Reference throughout this specification to features, advantages, orsimilar language does not imply that all of the features and advantagesthat may be realized with the present invention should be or are in anysingle embodiment of the invention. Rather, language referring to thefeatures and advantages is understood to mean that a specific feature,advantage, or characteristic described in connection with an embodimentis included in at least one embodiment of the present invention. Thus,discussions of the features and advantages, and similar language,throughout this specification may, but do not necessarily, refer to thesame embodiment.

Furthermore, the described features, advantages, and characteristics ofthe invention may be combined in any suitable manner in one or moreembodiments. One skilled in the relevant art will recognize, in light ofthe description herein, that the invention can be practiced without oneor more of the specific features or advantages of a particularembodiment. In other instances, additional features and advantages maybe recognized in certain embodiments that may not be present in allembodiments of the invention.

Reference throughout this specification to “one embodiment,” “anembodiment,” or similar language means that a particular feature,structure, or characteristic described in connection with the indicatedembodiment is included in at least one embodiment of the presentinvention. Thus, the phrases “in one embodiment,” “in an embodiment,”and similar language throughout this specification may, but do notnecessarily, all refer to the same embodiment.

FIG. 1 depicts an operational metric analysis system 100 in accordancewith an embodiment of the invention. The operational metric analysissystem is configured to perform an operational metric analysis for avirtual appliance to generate predictions or estimates regardingresource requirements and/or application performance metrics of thevirtual appliance. A virtual appliance is a pre-packaged software clientimage (e.g., a virtual machine image) that can be run on differentvirtualization platforms in public, private and hybrid cloudenvironments. In some embodiments, a virtual appliance is a virtualmachine image with a specific guest operating system (OS). For example,a virtual appliance may be a virtual machine image from which apreconfigured Linux virtual machine or a Windows virtual machine can beinstantiated. In some embodiments, a virtual appliance includes asoftware component/stack and metadata that contains informationregarding resource requirements of the virtual appliance, e.g., amountof memory and/or number of GHz in processor frequency desired for thevirtual appliance, and/or application performance metrics of the virtualappliance. As an example, the operational metric analysis system can beused to generate the information to be included in the metadata of thevirtual appliance.

In some embodiments, the operational metric analysis system 100 isconfigured to obtain application operational data from multipleinstances of a particular virtual appliance and generate an operationalmetric prediction for the virtual appliance based on the applicationoperational data. In some embodiments, an instance of a virtualappliance is a software client, e.g., a virtual machine (VM), which mayimplement various guest operating systems (OSs). In some embodiments,the operational metric prediction is used as guidance for futuredeployments of an instance of the virtual appliance in variousdeployment environments. Using operational data from multiple instancesof the same virtual appliance in different deployment environments, theoperational metric analysis system can cope with noises and disparitiesintroduced by the different deployment environments. The operationalmetric analysis system can detect and diagnose performance anomaliesusing application operational data from multiple instances of the samevirtual appliance and estimate resource usage and applicationperformance to make better provisioning and consolidation decisions.

In the embodiment depicted in FIG. 1, the operational metric analysissystem 100 is configured to obtain application operational data from atleast one instance 106-1 of a virtual appliance running in a privatecloud computing environment 102 and at least one instance 106-2 of thevirtual appliance running in a public cloud computing environment 104.In some embodiments, the private cloud computing environment is a cloudcomputing environment that has restricted access and the public cloudcomputing environment is a cloud computing environment that has openaccess. The exact number of computing environments from which theoperational metric analysis system obtains application operational datacan be any number of computing environments. In addition, the number ofinstances of the virtual appliance that can be operated on a singlecomputing environment (e.g., the private cloud computing environment orthe public cloud computing environment) can be any number of instances.The operational metric analysis system can generate an operationalmetric prediction for the virtual appliance based on the applicationoperational data obtained from the virtual appliance instances runningon the different cloud computing environments.

Turning now to FIG. 2, a distributed computer system 200 that canprovide a computing environment, such as the private cloud computingenvironment 102 or the public cloud computing environment 104, inaccordance with an embodiment of the invention is shown. The distributedcomputer system can be used to run one or more instances of a virtualappliance. As shown in FIG. 2, the distributed computer system includesa network 202, clusters C-1, C-2 . . . C-N of host computers (where N isa positive integer), storage 204 and a management computer 206. Theexact number of host computer clusters included in the distributedcomputer system can be any number of clusters from one to tens ofclusters or more. The host computers of the different clusters, thestorage and the management computer are connected to the network. Thus,each of the host computers in the clusters and the management computerare able to access the storage via the network and may share theresources provided by the storage. Consequently, any process running onany of the host computers and the management computer may also accessthe storage via the network.

In the illustrated embodiment, each of the clusters C-1, C-2 . . . C-Nincludes a number of host computers H-1, H-2 . . . H-M (where M is apositive integer). The host computers can be assigned to the hostcomputer clusters based on predefined criteria, which may includegeographical and/or logical relationships between the host computers.The number of host computers included in each of the clusters can be anynumber from one to several hundred or more. In addition, the number ofhost computers included in each of the clusters can vary so thatdifferent clusters can have different number of host computers. The hostcomputers are physical computer systems that host or support one or moreclients so that the clients are executing on the physical computersystems. As used herein, the term “client” is any software entity thatcan run on a computer system, such as a software application, a softwareprocess or a virtual machine (VM). The host computers may be serversthat are commonly found in data centers. As an example, the hostcomputers may be servers installed in one or more server racks.Typically, the host computers of a cluster are located within the sameserver rack.

Turning now to FIG. 3, components of a host computer 300 that isrepresentative of the host computers H-1, H-2 . . . H-M in the clustersC-1, C-2 . . . C-N in accordance with an embodiment of the invention areshown. In FIG. 3, the physical connections between the variouscomponents of the host computer are not illustrated. In the illustratedembodiment, the host computer is configured to support a number ofclients 320A, 320B . . . 320L (where L is a positive integer), which areVMs. The number of VMs supported by the host computer can be anywherefrom one to more than one thousand. The exact number of VMs supported bythe host computer is only limited by the physical resources of the hostcomputer. The VMs share at least some of the hardware resources of thehost computer, which include system memory 322, one or more processors324, a storage interface 326, and a network interface 328. The systemmemory, which may be random access memory (RAM), is the primary memoryof the host computer. The processor can be any type of a processor, suchas a central processing unit (CPU) commonly found in a server. Thestorage interface is an interface that allows that host computer tocommunicate with the storage 204. As an example, the storage interfacemay be a host bus adapter or a network file system interface. Thenetwork interface is an interface that allows the host computer tocommunicate with other devices connected to the network 202. As anexample, the network interface may be a network adapter.

In the illustrated embodiment, the VMs 320A, 320B . . . 320L run on topof a hypervisor 330, which is a software interface layer that enablessharing of the hardware resources of the host computer 300 by the VMs.However, in other embodiments, one or more of the VMs can be nested,i.e., a VM running in another VM. For example, one of the VMs may berunning in a VM, which is also running in another VM. The hypervisor mayrun on top of the host computer's operating system or directly onhardware of the host computer. With the support of the hypervisor, theVMs provide virtualized computer systems that give the appearance ofbeing distinct from the host computer and from each other. Each VMincludes a guest operating system (OS) 332 and one or more guestapplications (APP) 334. The guest operating system is a master controlprogram of the respective VM and, among other things, the guestoperating system forms a software platform on top of which the guestapplications run.

Similar to any other computer system connected to the network 202, theVMs 320A, 320B . . . 320L are able to communicate with other computersystems connected to the network using the network interface 328 of thehost computer 300. In addition, the VMs are able to access the storage204 using the storage interface 326 of the host computer.

Turning back to FIG. 2, the network 202 can be any type of computernetwork or a combination of networks that allows communications betweendevices connected to the network. The network may include the Internet,a wide area network (WAN), a local area network (LAN), a storage areanetwork (SAN), a fibre channel network and/or other networks. Thenetwork may be configured to support protocols suited for communicationswith storage arrays, such as Fibre Channel, Internet Small ComputerSystem Interface (iSCSI), Fibre Channel over Ethernet (FCoE) andHyperSCSI.

The storage 204 is used to store data for the host computers H-1, H-2 .. . H-M of the clusters C-1, C-2 . . . C-N, which can be accessed likeany other storage device connected to computer systems. In anembodiment, the storage can be accessed by entities, such as clients(e.g., VMs) running on the host computers, using any file system, e.g.,virtual machine file system (VMFS) or network file system (NFS). Thestorage includes one or more computer data storage devices 210, whichcan be any type of storage devices, such as solid-state devices (SSDs),hard disks or a combination of the two. The storage devices may operateas components of a network-attached storage (NAS) and/or a storage areanetwork (SAN). The storage includes a storage managing module 212, whichmanages the operation of the storage. In an embodiment, the storagemanaging module is a computer program executing on one or more computersystems (not shown) of the storage. The storage supports multipledatastores DS-1, DS-2 . . . DS-X (where X is an integer), which may beidentified using logical unit numbers (LUNs). In an embodiment, thedatastores are virtualized representations of storage facilities. Thus,each datastore may use the storage resource from more than one storagedevice included in the storage. The datastores are used to store dataassociated with the clients supported by the host computers of theclusters. For virtual machines, the datastores may be used to storevirtual storage, e.g., virtual disks, used by each of the virtualmachines, as well as other files needed to support the virtual machines.One or more datastores may be associated with one or more hostcomputers. Thus, each host computer is associated with at least onedatastore. Some of the datastores may be grouped into one or moreclusters of datastores, which are commonly referred to as storage pods.

The management computer 206 operates to manage the host computers H-1,H-2 . . . H-M of the clusters C-1, C-2 . . . C-N and/or the storage 204of the computer system 200. In some embodiments, the management computermay be implemented as a VMware® vCenter™ server (vCenter or VC) thatprovides centralized management of virtualized hosts and virtualmachines (“VMware” and “vCenter” are trademarks of VMware, Inc.). AvCenter can manage a vCenter server virtual appliance (VCVA), which is aself-contained virtual machine image that can be deployed and run as avirtual machine on a VMware® ESX® hypervisor (“VMware” and “ESX” aretrademarks of VMware, Inc.). In some embodiments, the managementcomputer is configured to generate, modify and/or monitor resourceconfigurations of the host computers and the clients running on the hostcomputers, for example, virtual machines (VMs). The configurations mayinclude hardware configuration of each of the host computers, such asCPU type and memory size, and/or software configurations of each of thehost computers, such as operating system (OS) type and installedapplications or software programs. The configurations may also includeclustering information, i.e., which host computers are included in whichclusters. The configurations may also include client hostinginformation, i.e., which clients, e.g., VMs, are hosted or running onwhich host computers. The configurations may also include clientinformation. The client information may include size of each of theclients, virtualized hardware configuration of each of the clients, suchas virtual CPU type and virtual memory size, software configuration ofeach of the clients, such as OS type and installed applications orsoftware programs running on each of the clients, and virtual storagesize for each of the clients. The client information may also includeresource settings, such as limit, reservation, entitlement and sharevalues for various resources, e.g., CPU, memory, network bandwidth andstorage, which are consumed by the clients. In an embodiment, themanagement computer may also be configured to generate, modify and/ormonitor the current configuration of the storage 204, including thephysical storage devices 210 and the datastores DS-1, DS-2 . . . DS-X ofthe storage.

Turning now to FIG. 4, a block diagram of components of the operationalmetric analysis system 100 in accordance with an embodiment of theinvention is shown. As illustrated in FIG. 4, the operational metricanalysis system 100 includes a data pre-processing unit 402, a featureidentification unit 404, a feature data analysis unit 406, anoperational metric prediction unit 408 and a prediction confidencefactor generation unit 410. These components of the operational metricanalysis system can be implemented as software, hardware or acombination of software and hardware. These components of theoperational metric analysis system may be located in the same computersystem or distributed across different computer systems.

Application operational data from multiple instances of a virtualappliance, e.g., a VCVA, deployed in different computing environmentsmay be organized in log files of the instances of the virtual appliance.In some embodiments, each instance (e.g., a virtual machine) of thevirtual appliance may produce one or more profiler logs, which provideinformation regarding the operational activities with respect to aparticular instance of the virtual appliance. The profiler logs maycontain, for example, performance metric information (e.g., memory usageinformation, processor (e.g., central processing unit (CPU)) usageinformation etc.), information regarding inventory (e.g., number ofvirtual machines), and/or information regarding operation activities(e.g., the frequency and duration of actions, such as, powering on/off avirtual machine, cloning virtual machines, etc.). In some embodiments,the operational metric analysis system 100 is configured to collect andanalyze profiler logs from different instances of the same virtualappliance running in different computing environments. The differentinstances of the virtual appliance may be run on the same operatingsystem or on different operating systems, such as, Windows and Linux.Each profiler log may include a set of workloads that are recorded fordifferent runs of a virtual appliance on different operating systems.

The application operational data from multiple instances of the virtualappliance may be in a compressed form to reduce data storagerequirements. In some embodiments, the data pre-processing unit 402 isconfigured to use one or more software tools (e.g., Python and/or zgrep)to decompress and extract raw features from compressed applicationoperational data.

The application operational data from multiple instances of the virtualappliance may include data of one or more operational features of theseinstances of the virtual appliance, which is classified in multiplecategories. FIG. 5 illustrates a list of categories of operationalfeatures of multiple instances of a virtual appliance. As shown in FIG.5, each category, “ProcessStats,” “InventoryStats,” “SessionStats,” or“RateCounter,” contains one or more operational features. The category,“ProcessStats,” of operational feature or features contains performancemetrics of interest. The operational feature or features that can beincluded in the category, “ProcessStats,” includes, for example,physical memory usage and user CPU usage. The number of operationalfeatures that can be included in the category, “ProcessStats,” may be 10or other suitable number. The category, “InventoryStats,” of operationalfeature or features contains inventory related metrics. The operationalfeature or features that can be included in the category,“InventoryStats,” includes, for example, number of computer clusters andnumber of virtual machines. The number of operational features that canbe included in the category, “InventoryStats,” may be 10 or othersuitable number. The category, “SessionStats,” of operational feature orfeatures contains session related metrics. The operational feature orfeatures that can be included in the category, “SessionStats,” includes,for example, number of sessions. The number of operational feature orfeatures that can be included in the category, “SessionStats,” may be 1or other suitable number. The category, “RateCounter,” of operationalfeature or features contains rate counters with a fixed interval (e.g.,5-minute). The operational feature or features that can be included inthe category, “RateCounter,” includes, for example, FilterCreates andFilterDestroys. In some embodiments, FilterCreate is an operation for aclient/user to inform a management server what events or properties theclient/user is interested in such that the client/user is notifiedwhenever there are any updates or changes to the events or properties.In some embodiments, FilterDestroys is an operation for removing apreviously specified event, property selection or update-trackingrequest. The number of operational features that can be included in thecategory, “RateCounter,” may be 30 or other suitable number.

Turning back to FIG. 4, the operational features of multiple instancesof the virtual appliance may be redundant and/or irrelevant to one ormore operational metrics of interest. In some embodiments, the featureidentification unit 404 is configured to identify operational featuresthat are relevant to an operational metric of the virtual appliance. Insome embodiments, the feature identification unit uses an entropy-basedmeasure, model or scheme to identify features that are relevant to oneor more operational metrics of interest. The operational metric analysissystem 100 can generate an operational metric prediction for the virtualappliance based on the identified operational features.

In some embodiments, the feature identification unit 404 is configuredto use an entropy-based measure, model or scheme that is based on mutualinformation to identify features that are relevant to one or moreoperational metrics of interest. Mutual information is a measure of theinformation that members of a set of random variables have on otherrandom variables in the same set. In some embodiments, mutualinformation I(X1, . . . , Xn) can be expressed as:I(X1, . . . ,Xn)=Σ_(i=1) ^(n) H(X _(i))−H(X),  (1)where Xi represents an operational metric of interest, X={X1, . . . ,Xn} and H(X) represents the entropy/uncertainty of X. Mutual informationof a set of random variables is 0 if the random variables in the set areindependent. Mutual information of a set of random variables is 0 if therandom variables in the set are not independent. In some embodiments,the feature identification unit is configured to establish aquantitative criterion for selecting features in a two-step process. Inone embodiment, the feature identification unit first identifiesfeatures, j, for which I(Xi, j)>1, where Xi is an operational metric ofinterest and j is a candidate feature. In this embodiment, the featureidentification unit subsequently identifies features, k, where themutual information between a performance metric of interest, Xi and afeature, k, is greater than the average mutual information of all thefeatures, j, that have a mutual information value greater than 1. Inthis embodiment, the mutual information between a performance metric ofinterest, Xi and a feature, k can be expressed as:

$\begin{matrix}{{{I\left( {X_{i},k} \right)} > {\frac{1}{n}{\sum\limits_{j = 1}^{n}\left( {{I\left( {X_{i},j} \right)} > 1} \right)}}},} & (2)\end{matrix}$where I(X_(i), k) represents mutual information between a performancemetric of interest, Xi and the feature, k, while I(X_(i), j) representsmutual information between a performance metric of interest, Xi and thefeature, j. By using the mutual information based measure to identifyfeatures that are relevant to an operational metric of interest, thefeature identification unit may not rely on the average mutualinformation over all the candidate metrics to identify features that arerelevant to one or more operational metrics of interest. Because thefeatures may be redundant or irrelevant to an operational metric ofinterest, identify features that are relevant to an operational metricof interest based on mutual information does not cause more featuresthan required, compared to using the average mutual information value.

In some embodiments, the feature data analysis unit 406 is configured toperform a data analysis on features that are identified as beingrelevant to one or more operational metrics of interest. The featuredata analysis unit may process features that are identified as beingrelevant to one or more operational metrics of interest to determine thediversity of the identified features and/or reduce the dimensions of theidentified features. In some embodiments, the feature data analysis unitis configured to perform a Principal Component Analysis (PCA) onfeatures that are identified as being relevant to one or moreoperational metrics of interest to provide a simpler representation ofthe features. The feature data analysis unit can perform a PCA toproject features from a higher dimensional space to a lower dimensionalspace.

In some embodiments, the operational metric prediction unit 408 isconfigured to generate an operational metric prediction for the virtualappliance. In an embodiment, the operational metric prediction unitgenerates an operational metric prediction for the virtual appliancebased on operational features identified by the feature identificationunit 404 and processed by the feature data analysis unit 406. Theoperational metric prediction for the virtual appliance may include aprediction or an estimation of an application resource metric and/or aprediction or an estimation of an application performance metric of thevirtual appliance. In some embodiments, the operational metricprediction for the virtual appliance may include at least of a physicalmemory usage of the virtual appliance, an average latency of the virtualappliance and a throughput of the virtual appliance.

In some embodiments, the operational metric prediction unit 408 uses aprediction model that is specific to an operating system platform togenerate an operational metric prediction for the virtual appliance. Insome embodiments, the operational metric prediction unit uses aprediction model for Windows operating system platform and uses adifferent prediction model for Windows operating system platform.

In some embodiments, the operational metric prediction unit 408 uses anearest-neighbor model that uses structure similarity betweenoperational features to make operational metric predictions. Theoperational metric prediction unit may use K-nearest neighbors (kNN) inwhich a prediction for an operational metric of interest is a functionof the k-closest points to a target point in the feature space. In someembodiments, the operational metric prediction unit uses a linearregression model, a support vector machines (SVMs) model or adecision/regression tree to make operational metric predictions.

In some embodiments, the operational metric prediction unit 408 isconfigured to build and train a model using data from n−1 of thedatasets and predict an operational metric of the excluded (held-out)dataset if there is structural similarity between the feature datasets.Disparities in the characteristics of individual feature datasets alsoinfluence the choice of models. In a regression based model, disparitiesin the characteristics of individual feature datasets can significantlybias the predictions. However, using a structural/similarity basedmodel, e.g., k Nearest Neighbors, where a prediction is made based onthe distance between data points, can lessen the extent to which adistinct dataset skews the predictions.

In some embodiments, the prediction confidence factor generation unit410 is configured to determine a confidence factor in an operationalmetric prediction for the virtual appliance. The confidence factor canbe used to adjust the operation of the feature identification unit 404,the operation of the feature data analysis unit 406, and/or theoperation of the operational metric prediction unit 408.

In some embodiments, the prediction confidence factor generation unit410 determines a confidence factor in an operational metric predictionusing a distance-based measure, model or scheme. The predictionconfidence factor generation unit may identify a low-confidenceprediction from a k-nearest neighbors' model by checking relativedistances between a particular point representing a prediction value andthe k-nearest neighbors of the particular point that represent similarprediction values. Distance-based measure can fare well onlow-dimensional data obtained from a pre-processing step ofdimensionality reduction via techniques such as Principal ComponentAnalysis (PCA). The prediction confidence factor generation unit may usethe k-nearest, the k−1 nearest or a majority of the nearest neighbors.In one embodiment, the prediction confidence factor generation unitidentifies an operational metric prediction as a low-confidenceprediction if the standard deviation over the distances from a new pointto each of the set of neighbors is larger than a threshold.

In some embodiments, the prediction confidence factor generation unit410 determines a confidence factor in an operational metric predictionusing a statistical cluster membership inclusion/exclusion measure,model or scheme. The prediction confidence factor generation unit maycombine the use of k-nearest neighbors and k-means clustering models.K-means clustering can be used to identify groups of similar points byidentifying groupings of points around centroids of concentration. Theprediction confidence factor generation unit may identify that anoperational metric prediction is a function of the k-nearest neighborsin the same k-means cluster. The prediction confidence factor generationunit may determine whether the point for which a prediction fallsoutside (or is at the edge) of the k-means cluster it is most likely tobe a member of by comparing its distance from the cluster centroid withthe distances of all the other points in the cluster from the centroid.The prediction confidence factor generation unit may identify alow-confidence prediction if a point tending towards the edge of acluster.

In some embodiments, the prediction confidence factor generation unit410 calculates a statistical recall factor for predictions of anoperational metric. In some embodiments, a recall factor for predictionsof an operational metric is expressed as:

$\begin{matrix}{{{Recall} = \frac{truepositives}{{truepositives} + {falsenegatives}}},} & (3)\end{matrix}$where Recall represents the recall factor, truepositives represents thenumber of truepositives in the predictions of the operational metric,and falsenegatives represents the number of false-negatives in thepredictions of the operational metric. A true-positive may be deemed asone prediction where a predicted operational metric (e.g., physicalmemory usage) is greater than or equal to the observed value of theoperational metric. A false-negative may be deemed as one predictionwhere a predicted operational metric (e.g., physical memory usage) isless than the observed value of the operational metric. A high recallfactor indicates that the operational metric analysis system 100overestimates an operational metric while a low recall factor indicatesthat the operational metric analysis system underestimates anoperational metric. In some embodiments, the prediction confidencefactor generation unit 410 calculates a Root Mean Squared Error (RMSE)for predictions of an operational metric.

FIG. 6 is a flow chart that illustrates an exemplary operation of theoperational metric analysis system 100. The operational metric analysissystem begins operation, at step 600 and stops operation, at step 612.The data pre-processing unit 402 decompresses and extracts applicationoperational features from compressed log files of multiple instances ofa virtual appliance, at step 602. The feature identification unit 404identifies operational features that are relevant to an operationalmetric of the virtual appliance, at step 604. The feature data analysisunit 406 processes operational features that are identified as beingrelevant to the operational metric to reduce the dimensions of theidentified features, at step 606. The operational metric prediction unit408 generates an operational metric prediction for the virtual appliancebased on the processed operational features, at step 608. The predictionconfidence factor generation unit 410 determines a confidence factor forthe operational metric prediction, at step 610. The generatedoperational metric prediction and confidence factor can then be includedin the metadata of the virtual appliance. This information may be usedto properly allocate sufficient resources for each instance of thevirtual appliance and/or determine the number of instances of thevirtual appliance to deploy.

A method for performing an operational metric analysis for a virtualappliance in accordance with an embodiment of the invention is describedwith reference to a flow diagram of FIG. 7. At block 702, applicationoperational data is obtained from multiple instances of the virtualappliance. At block 704, an operational metric prediction for thevirtual appliance is generated based on the application operationaldata.

Although the operations of the method(s) herein are shown and describedin a particular order, the order of the operations of each method may bealtered so that certain operations may be performed in an inverse orderor so that certain operations may be performed, at least in part,concurrently with other operations. In another embodiment, instructionsor sub-operations of distinct operations may be implemented in anintermittent and/or alternating manner.

It should also be noted that at least some of the operations for themethods may be implemented using software instructions stored on acomputer useable storage medium for execution by a computer. As anexample, an embodiment of a computer program product includes a computeruseable storage medium to store a computer readable program that, whenexecuted on a computer, causes the computer to perform operations, asdescribed herein.

Furthermore, embodiments of at least portions of the invention can takethe form of a computer program product accessible from a computer-usableor computer-readable medium providing program code for use by or inconnection with a computer or any instruction execution system. For thepurposes of this description, a computer-usable or computer readablemedium can be any apparatus that can contain, store, communicate,propagate, or transport the program for use by or in connection with theinstruction execution system, apparatus, or device.

The computer-useable or computer-readable medium can be an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system(or apparatus or device), or a propagation medium. Examples of acomputer-readable medium include a semiconductor or solid state memory,magnetic tape, a removable computer diskette, a random access memory(RAM), a read-only memory (ROM), a rigid magnetic disc, and an opticaldisc. Current examples of optical discs include a compact disc with readonly memory (CD-ROM), a compact disc with read/write (CD-R/W), a digitalvideo disc (DVD), and a Blu-ray disc.

In the above description, specific details of various embodiments areprovided. However, some embodiments may be practiced with less than allof these specific details. In other instances, certain methods,procedures, components, structures, and/or functions are described in nomore detail than to enable the various embodiments of the invention, forthe sake of brevity and clarity.

Although specific embodiments of the invention have been described andillustrated, the invention is not to be limited to the specific forms orarrangements of parts so described and illustrated. The scope of theinvention is to be defined by the claims appended hereto and theirequivalents.

What is claimed is:
 1. A method for performing an operational metricanalysis for a virtual appliance, comprising: obtaining applicationoperational data from a first plurality of instances of the virtualappliance by identifying one or more operational features associatedwith an operational metric of the virtual appliance using anentropy-based model, wherein the entropy-based model is based upon ameasure of how much information is obtained about the operational metricthrough the one or more operational features and the entropy-based modelis based on a mutual information calculation representing an amount ofmutual information between the operational metric and the applicationoperational data; generating an operational metric prediction for thevirtual appliance based on the application operational data; allocatingresources for each instance of a second plurality of instances of thevirtual appliance based on the operational metric prediction.
 2. Themethod of claim 1, wherein the application operational data comprises atleast one of application resource usage data and application performancedata.
 3. The method of claim 1, wherein the application operational datacomprises a plurality of operational features of the instances of thevirtual appliance.
 4. The method of claim 3, further comprisingidentifying operational features that are relevant to an operationalmetric of the virtual appliance using the entropy-based model.
 5. Themethod of claim 4, wherein generating the operational metric predictionfor the virtual appliance comprises generating the operational metricprediction for the virtual appliance based on the identified operationalfeatures using a nearest-neighbor model.
 6. The method of claim 4,wherein identifying operational features that are relevant to theoperational metric of the virtual appliance using the entropy-basedmodel comprises identifying operational features that are relevant tothe operational metric of the virtual appliance based on mutualinformation of the application operational data.
 7. The method of claim1, wherein the operational metric prediction for the virtual appliancecomprises at least one of a physical memory usage of the virtualappliance, an average latency of the virtual appliance and a throughputof the virtual appliance.
 8. A non-transitory computer-readable storagemedium containing program instructions for performing an operationalmetric analysis for a virtual appliance, wherein execution of theprogram instructions by one or more processors causes the one or moreprocessors to at least: obtaining application operational data from afirst plurality of instances of the virtual appliance by identifying oneor more operational features associated with an operational metric ofthe virtual appliance using an entropy-based model, wherein theentropy-based model is based upon a measure of how much information isobtained about the operational metric through the one or moreoperational features and the entropy-based model is based on a mutualinformation calculation representing an amount of mutual informationbetween the operational metric and the application operational data;generating an operational metric prediction for the virtual appliancebased on the application operational data; and allocating resources foreach instance of a second plurality of instances of the virtualappliance based on the operational metric prediction.
 9. Thenon-transitory computer-readable storage medium of claim 8, wherein theapplication operational data comprises at least one of applicationresource usage data and application performance data.
 10. Thenon-transitory computer-readable storage medium of claim 8, wherein theapplication operational data comprises data of a plurality ofoperational features of the instances of the virtual appliance, andwherein the data of the operational features of the instances of thevirtual appliance is classified in a plurality of categories.
 11. Thenon-transitory computer-readable storage medium of claim 10, wherein thesteps further comprise identifying operational features that arerelevant to an operational metric of the virtual appliance using theentropy-based model.
 12. The non-transitory computer-readable storagemedium of claim 11, wherein generating the operational metric predictionfor the virtual appliance comprises generating the operational metricprediction for the virtual appliance based on the identified operationalfeatures using a nearest-neighbor model.
 13. The non-transitorycomputer-readable storage medium of claim 11, wherein identifyingoperational features that are relevant to the operational metric of thevirtual appliance using the entropy-based model comprises identifyingoperational features that are relevant to the operational metric of thevirtual appliance based on mutual information of the applicationoperational data.
 14. The non-transitory computer-readable storagemedium of claim 8, wherein the operational metric prediction for thevirtual appliance comprises at least one of a physical memory usage ofthe virtual appliance, an average latency of the virtual appliance and athroughput of the virtual appliance.
 15. A system for performing anoperational metric analysis for a virtual appliance comprising: aprocessor; and a memory storing program code, which, when executed onthe processor, performs the operational metric analysis for the virtualappliance, comprising: obtaining application operational data from afirst plurality of instances of the virtual appliance by identifying oneor more operational features associated with an operational metric ofthe virtual appliance using an entropy-based model, wherein theentropy-based model is based upon a measure of how much information isobtained about the operational metric through the one or moreoperational features and the entropy-based model is based on a mutualinformation calculation representing an amount of mutual informationbetween the operational metric and the application operational data;generating an operational metric prediction for the virtual appliancebased on the application operational data; allocating resources for eachinstance of a second plurality of instances of the virtual appliancebased on the operational metric prediction.
 16. The system of claim 15,wherein the application operational data comprises at least one ofapplication resource usage data and application performance data. 17.The system of claim 15, wherein the application operational datacomprises data of a plurality of operational features of the instancesof the virtual appliance, and wherein the data of the operationalfeatures of the instances of the virtual appliance is classified in aplurality of categories.
 18. The system of claim 17, wherein theoperational metric analysis further comprises identifying operationalfeatures that are relevant to an operational metric of the virtualappliance using the entropy-based model.
 19. The system of claim 18,wherein generating the operational metric prediction for the virtualappliance comprises generating the operational metric prediction for thevirtual appliance based on the identified operational features using anearest-neighbor model.
 20. The system of claim 18, wherein identifyingoperational features that are relevant to the operational metric of thevirtual appliance using the entropy-based model comprises identifyingoperational features that are relevant to the operational metric of thevirtual appliance based on mutual information of the applicationoperational data.